1. Field of the Invention
The present invention generally relates to the art and technology of machine learning and data mining and, more particularly, to outlier detection methods and apparatus which are light on computational resources without sacrificing predictive performance.
2. Background Description
There are certain applications of data mining, for example, in which it is required to classify future examples into normal and abnormal classes based on past data that consist only of normal data. This type of problem is called “outlier detection” or “anomaly detection” and is a form of the so-called unsupervised learning in which no labeled data are available at the training time. Real world problems for which outlier detection is desired include intrusion detection in network data and fraud detection in telecom and financial data.
To date, most existing methods of outlier detection are based on density estimation methods, that is, methods that attempt to estimate the distribution of the normal data, and then use the estimated density model to calculate the probability of each test data, and classify those that receive unusually small probabilities as outliers, and thus as candidates of the class of interest, such as intrusions or frauds.
With this type of method based on density estimation, there has been a problem that the required computational resources are heavy, both in terms of computation time and storage. For example, a representative density estimation method, known as the Parzen window method, estimates the density as the mixture of Gaussians placed around all training data. See, Yueng and Chow, “Parzen-Window Network Intrusion Detectors”, Proc. of the 16th International Conference on Pattern Recognition, 2002. As such, it requires the storage of all input data points, thus necessitating an amount of storage that grows linearly in the number of data.
In many applications of outlier detection, such as intrusion and fraud detection, the number of data tends to be extremely large, say in tens of millions, and computationally demanding methods are not well-suited, especially those based on storing all past data.